A Novel Approach to Integrated Search Information Retrieval Technique for Hidden Web for Domain Specific Crawling

The traditional web crawlers retrieve contents from only the “Surface web” and are unable to crawl through the hidden portion of the Web containing high quality information which is dynamically generated through querying databases when the queries are submitted through a search interface. For Hidden web, most of the published research has been done to identify/detect such searchable forms and make a systematic search over these forms. One approach is based on a Web crawler that identifies, analyzes search forms and fills them with appropriate content to retrieve maximum relevant information from the deep web. A critical problem with this is the search interface integration followed by the integration of the results obtained as a result of parallel request quest. This paper proposes a technique to detect and construct an integrated query interface that integrates a set of web interfaces over a given domain of interest. It provides users to access information uniformly from multiple sources. The proposed strategy does that by focusing the crawl on a given topic; by judiciously analyzing the domain specific matching, mapping information/key attributes which leads to web pages containing domain specific search forms in integrated manner. The interface mapping and matching library, semantic knowledgebase specific to the domain play a very key role here and demand for continuous evolution, improvement to make hidden web crawling effective and usable.